Construct Validation of the Rainbow Model of Integrated Care Measurement Tool in Dutch Primary Care for Older Adults

Introduction: Care integration in primary elderly care is suboptimal. Validated instruments are needed to enable the implementation of integrated primary care. We aimed to assess construct validity of the Rainbow Model of Integrated Care measurement tool (RMIC-MT) for healthcare professionals working in an integrated primary elderly care setting in the Netherlands. Methods: In a cross-sectional study, the RMIC-MT, a 36-item questionnaire covering all domains of the Rainbow Model of Integrated Care (RMIC), was sent out to local networks of primary elderly care professionals. Confirmatory factor analysis with maximum likelihood estimation was used for the validation of the factor structure of the RMIC-MT. Model fit was assessed by the chi-square test and fit indices. Results: The RMIC-MT was completed by 323 professionals, primarily general practitioners, community nurses, practice nurses, and case managers. Confirmatory factor analysis and corresponding fit indices showed moderate to good fit, thereby confirming a nine factor model with a total of 36 items. Conclusions: The RMIC-MT is promising for the primary elderly care setting in the Netherlands. It can be used for evaluating integrated care initiatives in a primary care setting, thereby contributing to implementation of integrated primary elderly care.


INTRODUCTION
The number of older persons living at home with complex healthcare needs is rising. Currently, care for older persons is fragmented as a result of suboptimal communication and coordination among the multiple healthcare professionals that are often involved in a primary care setting [1][2][3]. It is considered essential to cross the disciplinary boundaries between professionals to enable care coordination between multiple healthcare services [2]. Integrated care is an important strategy to overcome fragmentation of care. A framework that captures the concept of integrated care and describes its key domains is the Rainbow Model of Integrated Care (RMIC) [4] that was specifically developed for evaluating integrated primary care settings. Improving care integration and thereby quality of care could contribute to a more sustainable, patient-friendly, and affordable health care system [5,6].
To enable implementation of integrated care for older people it is essential to measure integrated care performance in practice [7][8][9]. Nevertheless, validated measurement instruments are lacking [10][11][12][13][14], because of the complexity of integrated care as a construct [16]. Available instruments are thus either of poor psychometric quality [11,14] or are too elaborate for use in practice [15]. Based on the RMIC framework, the Rainbow Model of Integrated Care Measurement Tool (RMIC-MT) was developed [16]. This questionnaire was tested for internal consistency and construct validity internationally for renal care and in Chinese primary care [16,17]. Since the RMIC-MT is a valid measure to evaluate integrated care in several care settings, it may have the potential to validly measure the integration of elderly care networks and support professionals to improve their interprofessional collaboration and eventually the quality of care for older persons living at home. Therefore, we aim to assess the construct validity of the RMIC-MT for healthcare professionals working in an integrated primary elderly care setting in the Netherlands.

STUDY DESIGN
A cross-sectional study design was used for construct validation of the RMIC-MT for healthcare professionals in elderly care. Data were collected between February 2020 and February 2021.

SETTING AND STUDY POPULATION
Participants were healthcare professionals active in primary elderly care networks throughout the Netherlands. Networks were recruited via two routes: 1) via the DementiaNet study and 2) via newsletters of general practitioner-and geriatric specialists organisations and all regional elderly care networks in the Netherlands. The DementiaNet program [18] supports network development in primary elderly and dementia care. These networks include medical, care, and social professionals and provide interprofessional care to a shared caseload of persons with dementia. More information about the DementiaNet program is available elsewhere [18,19]. An overview of the networks' geographical location can be found in Appendix I.
We used convenience sampling with the following inclusion criteria for the primary elderly care networks: 1) at least a general practitioner practice and community nurse present, 2) professionals from at least two organisations present, 3) a network leader is appointed, 4) all network members share a caseload of at least two persons, 5) work agreements are made or a regular multidisciplinary meeting is held, and 6) at least three persons of the network are willing to complete the questionnaire. All networks meeting the inclusion criteria were invited to participate, independent of their development. When inclusion criteria were met, network members or the contact person were invited via email to participate in the study. All participating networks received a feedback report, enabling them to identify areas of improvement.

SAMPLE SIZE
To determine the a priori sample size we used the rule of thumb of including 10 subjects per questionnaire item [20]. Given that the RMIC-MT for professionals has 36 items our targeted sample size was 360 participants.

THEORETICAL FRAMEWORK
The RMIC is a validated framework that was developed to operationalise the construct of integrated primary care. Primary care is considered the focal point for integrating various medical, care, and welfare services close to people's homes. The RMIC defines the necessary integration mechanisms by also emphasizing its final (Triple aim) outcomes [4]. Different integration domains are specified in the RMIC, making it possible to identify specific areas for improvement. The RMIC describes three categories of integrated care: the scope, type, and enablers of integration, consisting of eight domains. The scope entails the person-focused and populationfocused view of health issues at the micro and mesomacro level respectively. The type of integration refers to four domains: clinical integration, professional integration, organisational integration, and system integration. Enablers entails functional and normative integration. Further explanation of the RMIC can be found in Appendix II.

RMIC MEASUREMENT TOOL
The RMIC-MT consists of a 36-item questionnaire for professionals [21], covering all aspects of integrated care as described by the RMIC [4]. The development and international validation of the RMIC-MT is described elsewhere [16]. The RMIC-MT was translated to Dutch using forward translation by an independent translator and adapted to the Dutch primary care setting for older adults. To optimize applicability in primary elderly care the Dutch RMIC-MT was reviewed and pilot-tested by six experts in the field of primary elderly care, including three authors (DO, MP, and MN) and three external reviewers with a healthcare professional (general practitioner, community nurse, and case manager) and managerial backgrounds. The version of the RMIC-MT used in this study can be found elsewhere.
The RMIC-MT has the following domains: Personcenteredness (e.g. needs assessment), populationcenteredness (e.g. population screening), clinical coordination (e.g. personal care plan), professional coordination (e.g. multidisciplinary team), organisational coordination (e.g. inter-organisational partnerships), system coordination (e.g. policy and financing), technical competence (e.g. interoperable medical records), triple aim (e.g. outcome assessment), and cultural competence (e.g. collaboration culture). We reformulated the name of the domain 'triple aim' to 'results-oriented', which better describes the items of this domain. The RMIC-MT consisted of 36 items divided over these nine integration domains (see Appendix III for the number of items per domain). The RMIC-MT assessed how formal providers of primary care for older adults perceived their network's ability to deliver integrated care on a five-point Likert scale (options were: don't agree at all, don't agree, neutral, agree, fully agree; I don't know or never, rarely, sometimes, often, always). In this study, we added the 'I don't know' option. At the end of the RMIC-MT questionnaire information on professional background was collected.

DATA COLLECTION
The RMIC-MT questionnaire was sent by email between February 2020 and February 2021. The RMIC-MT questionnaires were completed online using a web-based survey platform. Consent was asked at the beginning of the questionnaire. To prevent missing answers all questions were mandatory. The response rate per network was assessed through network-specific codes assigned to each questionnaire. Participants received a reminder after one week and after two weeks when the questionnaire was not completed.

ANALYSIS
Number and percentage were calculated for the background characteristics of the participants. For the final dataset we merged the 'I don't know' scores with the 'neutral' option, as the previous studies did not have an 'I don't know' option and both options express the lack of a directed opinion. We first performed an Exploratory factor analysis (EFA) to explore the underlying factor structure. Confirmatory factor analysis (CFA) was used to build upon the previous results of the RMIC-MT [16] that consisted of 36 items and nine factors [16,17].
Maximum likelihood estimation was used on the variance-covariance matrix and model fit was assessed using the chi-square statistic and goodness-of-fit indices. The fit indices that we used were the Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) (both considered adequate if above 0.9 and good if above 0.95), Root Mean Square Error of Approximation (RMSEA, good if close to 0.06 or below), and Standardized Root Mean Square Residual (SRMR, good if close to 0.08 or below) [22]. RMSEA is an absolute fit index that assesses the difference between the hypothesized and a perfect model [23]. CFI and TLI are incremental fit indices which compare the fit of the data and the hypothesized model [23]. The SRMR is the average of the standardized residuals between the covariance matrix of the data and the model [24]. These fit indices were considered in combination, as good model fit entails meeting all these criteria [22,25]. Significance of the chi-quare statistic indicates good model fit when the p-value is not significant (i.e. above .05). R Studio version 3.6.2 [26] with the Lavaan package [27] were used for analysis.

ETHICAL APPROVAL
The study was conducted according to the principles of the Declaration of Helsinki (2013). The research ethics committee of the Radboud university medical center stated that the study did not fall within the remit of the Medical Research Involving Human Subjects Act (WMO) (file number: 2019-5599).

PARTICIPANT CHARACTERISTICS
Three hundred twenty-three healthcare professionals started the survey, of which 282 completed the RMIC-MT. In total, 262 of 450 DementiaNet professionals participated (58% response rate), divided over 34 networks. Furthermore, 61 external professionals from 10 different networks participated. The majority of professionals were case managers (18%), community nurses (18%), general practitioners (12%), and practice nurses (11%). Median time needed for completing the RMIC-MT was 15 minutes (Inter Quartile Range = 13 minutes). Table 1 provides an overview of the participant characteristics.

MISSING VALUE PATTERN
Forty-one participants partly completed the RMIC-MT. Since all questions were set as required (see details in method section) these incomplete RMIC-MTs are all missing the answers to the later items. See Table 2 for an overview of the total number of responses per item. We used all the answers (as mentioned in Table 2) for the analysis, also from incomplete questionnaires.

CONFIRMATORY FACTOR ANALYSIS
The EFA results showed a similar factor structure compared to the previous studies [16,17] (Appendix IV). Therefore, we continued with the CFA. Table 3 shows the factor loadings of the RMIC-MT and Appendix V provides the item correlation matrix. The chi-square test was significant, indicating bad model fit: χ 2 (558) = 945.08, p < .001. Using the previously mentioned cut-off values, both RMSEA (0.046) and SRMR (0.055) are considered good, while CFI (0.895) and TLI (0.882) are slightly under their cut-off and are therefore considered moderate. Summary measures including median, mean, and standard deviations for all items are depicted in Table 2.

PRINCIPAL FINDINGS
This study provides the first assessment of the construct validation of the Dutch RMIC-MT version for professionals working in the integrated primary elderly care setting. In the CFA, the nine factors of the RMIC-MT passed the majority of goodness-to-fit tests. This makes the RMIC-MT's utility promising, but not yet ready to assess integrated care in daily practice.

COMPARISON WITH EXISTING EVIDENCE
We were able to build upon previous validations of the RMIC-MT in different countries, languages, and for different conditions [16,17,28]. Although we were not able to successfully validate the RMIC-MT construct in Dutch for the elderly care setting, we were able to contribute to the evidence of its applicability and representativeness in the primary integrated care setting.
We found moderate rather than good results for a few of our fit indices. This could be the result of our relatively small sample size compared to the previous successful validation studies of the RMIC [16,17], as fit indices tend to indicate inappropriate fit in smaller sample sizes [29]. We found a slightly different factor structure than previous studies. This may be, because our explained variance for some of the domains may have differed from the previously validated RMIC-MT. As previous RMIC-MT validation studies already showed, the explained variance between studies varied for e.g. the domain cultural competence and care integration [16,17], which are likely due to cultural differences between countries or the difference in target condition (e.g. renal care or mental health care).
With this study, we add to the evidence that the RMIC-MT questionnaire is a promising tool to measure primary elderly care integration. Valid easy-to-use instruments to measure integrated care in the primary elderly care setting are lacking and available tools are of poor quality [10][11][12][13][14]30]. Almost all other available tools only focus on a single aspect of care integration, mainly person-focused care and clinical integration and do not represent normative and system integration [11]. The RMIC-MT questionnaire is based on the RMIC framework, and therefore covers all relevant domains of integrated care, therewith doing justice to complexity of integrated care [15,16,31].
We observed that thirteen percent of professionals did not complete the questionnaire. Reasons could be that the RMIC-MT is perceived as too time consuming or professionals find it difficult to interpret the questions. The last explanation is also supported by the fact that professionals frequently answered 'I don't know/neutral'. Since for system coordination, professional coordination, and technical competence the majority of the answers were 'I don't know/neutral', we assume that the RMIC-MT questions for these topics might be outside the scope of the participating professionals. Extra explanatory notes might be necessary. Especially the answers given for system coordination can be explained, since professionals find it difficult to assess system domains of integrated care [4]. Perhaps system coordination policies are not experienced by professionals in daily practice   and out of their scope of influence and therefore not recognizable. We did not expect and cannot explain why professionals had difficulties with answering questions about professional coordination as this is obviously part of their daily practice. Previous research shows that professionals focus and improve most on this domain [32] when developing their care integration. Possibly and despite pilot-testing, the terminology or formulation used in the questionnaire to describe this domain may not have matched the vocabulary of the participating professionals. This should be clarified in future research.

STRENGTHS AND LIMITATIONS
One of the strengths of this study is that we could build upon previous results from RMIC-MT construct validations for various conditions in primary care, which solidifies the results for this Dutch version for primary elderly care [16,17]. Moreover, this will contribute to the comparability of the results for different settings by using the same measurement tool.
Another strength is that we were able to include networks of elderly care professionals throughout the Netherlands, even though a large part of the networks were DementiaNet networks located in the east of the Netherlands. The participants also had varying professional backgrounds with all relevant disciplines represented.
A limitation is our sample size. We had a high number of non-responders, reasons for which remained unclear. However, data was collected during times of COVID-19, which potentially hampered the responses, since professionals were mainly focused on direct patient care. We reached 90% of the intended sample size (i.e. 323 of 360 participants). Although, literature suggests that 300 cases is generally sufficient for a CFA [33], our participants frequently did not finish the questionnaire which decrease power. For the CFI and TLI fit indices the results were considered moderate, for which the cause might have been the relatively small sample size. The results of the chi-square test tend to be less reliable with a small sample size and the other fit indices also tend to show a worse fit [29]. We recommend to perform a study with a larger sample size to identify whether these issues are then solved. Another limitation is the combined analysis of the 'I don't know' option and  the 'neutral' option: evidently these answers do not exactly mean the same. Since these two options both similarly indicate that the professional had no strong preference, this will most likely not affect our results. In future research, we suggest to remove the 'I don't know' option.

IMPLICATIONS FOR RESEARCH AND PRACTICE
Our CFA of the RMIC-MT for Dutch primary elderly care is a valuable first step for application of the RMIC-MT in this setting in the Netherlands. The RMIC-MT can be used for performance assessment (e.g. network evaluations) of the collaboration processes [9,16] in primary care networks. These assessments can reveal areas for further improvement of their care integration according to the different domains of integrated care, for example improving interprofessional communication by using a digital tool or by implementing a multidisciplinary meeting [9,16]. Moreover, insights from the RMIC-MT assessments can be used for educational purposes, thereby tailoring trainings to the needs of a network. In research, RMIC-MT has the potential to become the preferred instrument for measurement of care integration, as, with the current study included, it was already validated in different primary care settings and for various conditions [11]. Validation of the RMIC-MT for primary elderly care should be repeated in a larger sample size, to hopefully show good fit indices. Subsequently, future studies may include assessment of its feasibility and its applicability for improvement purposes. Eventually, application in intervention studies may show whether if the tool indeed enables networks to improve their care integration.
Although the vast majority of professionals filled in the entire questionnaire, we did receive incomplete questionnaires. A shorter and easier formulated version already showed positive results for physical and mental healthcare [34]. Moreover, identifying if the questions that are frequently answered with 'I don't know' are not too difficult or irrelevant for this target group is essential. In this study, we focused on the RMIC-MT version for professionals. To give professionals an even more complete overview of their current care integration, incorporating multiple stakeholder perspectives' is needed. It is therefore desirable to also include patients' and informal caregivers' experiences, by e.g. using the RMIC-MT patient version in addition to the professional version [16,35,36].

CONCLUSION
We were able to build upon previous validations of the RMIC-MT and passed the majority of goodness-to-fit tests. This makes the RMIC-MT's utility promising for the primary elderly care setting in the Netherlands, but not yet ready to assess integrated care in daily practice. Feasibility and added value of the tool should be studied after this small-scale validation study. The instrument has the potential to facilitate care integration, as it eventually could be used for evaluating integrated care initiatives in a primary care setting, thereby contributing to implementation of integrated primary elderly care.

ADDITIONAL FILE
The additional file for this article can be found as follows: • Appendixes. Appendix I to Appendix V. DOI: https:// doi.org/10.5334/ijic.6739.s1